STILL PICTURE PROCESSING FOR MPEG-2 VIDEO 



BACKGROUND OF THE INVENTION 

This application claims the benefit of U.S. 
Provisional Patent Application No. 60/171,773, filed 
December 22, 1999. 

The present invention relates to digital video, 
and more particularly to a scheme for enabling still 
picture sequences (sometimes referred to as "still 
frame" sequences) to be communicated at a very low bit 
rate (e.g., under 200 Kbps) in the same transport 
stream that is carrying full motion video, multimedia 
and/or audio services such as conventional television 
programming. The techniques of the invention are 
particularly suitable, e.g., in providing still 
pictures for weather channels, album covers (e.g., 
text and graphics) that may accompany music channels, 
real estate sales channels, home shopping channels, 
and the like. For example, the invention will enable 
the album cover of a music video to be displayed while 
the video is playing, with the album cover video 
information being provided at a reduced bit rate (and 
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consequent savings in bandwidth) with respect to the 
full motion music video (i.e., "normal video"). 

One prior technique for providing still picture 
images in a digital television environment involves 
5 precompressing the still pictures, storing the 

precompressed frames, and then providing the stored 
precompressed frames for decompression and display 
when requested. This technique is not advantageous 
since it does not provide real-time processing of the 

10 frames and, as a result, requires additional hardware 

(such as a server) to store the precompressed files 
and deliver them when required. 

Another prior art technique is that used in the 
Digicipher® II digital television system developed by 

15 General Instrument Corporation of Horsham, 

Pennsylvania, U.S.A. In this system, video frames are 
normally compressed as Predictive (P) frames, and an 
Intra (I) frame is only processed when specifically 
requested by a syntax processor. In order to capture 

20 a still picture, the syntax processor instructs the 

video compressor to go into a still frame mode by 
setting a still frame mode flag. This causes the 
compressor frame buffer pointers to freeze, which in 
turn causes the same frame to be repeated. When a new 

25 frame is to be captured, the syntax processor first 
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disables the still frame mode flag to the compressor, 
and the compressor returns to a normal video mode 
operation. To capture a new still picture, the still 
frame flag is set again. This sequence of operation 
5 adds complexity due to the requirement to set and re- 
set the still frame mode flag and to switch the 
compressor between P-frame and I-frame processing. 

Accordingly, it would be desirable to provide 
simplified and less costly apparatus and methods for 

10 encoding and transmitting still picture images, and 

for transmitting such images at reduced frame and 
transmission rates, using the MPEG-2 or similar 
standard. It would be further advantageous to provide 
such apparatus and methods that can be implemented 

15 using a standard video compressor that operates in the 

same manner and using the same algorithms for both 
still pictures and normal video. It would be still 
further advantageous to provide such apparatus and 
methods that will operate with a conventional decoder, 

20 without the need for a special still frame decoder 

mode. Such a scheme would advantageously utilize the 
MPEG-2 "still picture mode" at the decoder. 

An advantageous system as described above would 
provide a bandwidth efficient and cost effective 

25 solution to the communication of still picture images 
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for digital television and the like. The present 
invention provides methods and apparatus having the 
aforementioned and other advantages. 
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SUMMARY OF THE INVENTION 

A method and apparatus are provided which enable 
still picture images to be encoded and transmitted at 
a reduced frame repetition rate (e.g., no more than 
5 about 1/15 to two frames per second) and a reduced 

transmission rate (e.g./ about 100 Kbps - 200 Kbps or 
less) using the Moving Picture Experts Group (MPEG-2) 
standard or other digital video standard. 

In accordance with the invention, a standard 

10 video compressor processes still video frames in the 
same manner that conventional full motion video is 
compressed. In order to maintain a standard 
throughput for the video compressor (e.g., at an input 
video frame rate) , while providing a reduced still 

15 picture repetition rate in the ultimate transport 

stream, a compressed video buffer (e.g., video FIFO) 
is provided to receive output from the video 
compressor. The compressed video buffer is gated 
(either at its input, its output, or both) to provide 

20 frames of compressed video data at the reduced still 

picture repetition rate. 

In one embodiment, which is particularly useful 
when still pictures are captured from an input video 
stream at a "capture rate" that is less than the input 



video frame rate in frames per second, both "current 
frames" and "previous frames" are provided as input to 
the compressor, with the previous frames being 
provided on an ongoing basis between the provision of 
current frames. The current frames are provided at 
the still picture capture rate. The previous frames 
are frames that are originally received as current 
frames, and stored in a buffer for subsequent output 
as previous frames. 

In an example embodiment, the still picture 
capture rate is between about one and five times the 
repetition period. For example, if the repetition 
period is one second, and the capture rate is three 
seconds, then one frame is sent every one second and a 
new still picture is captured every three seconds. 
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BRIEF DESCRIPTION OF THE DRAWING 

Figure 1 is a block diagram illustrating a still 
picture encoder in accordance with the invention; and 

Figure 2 is a block diagram illustrating an 
5 alternate embodiment of a still picture encoder in 

accordance with the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

The present invention enables still picture 
sequences to be communicated at a very low bit rate. 
As used herein, such a low bit rate can comprise, for 
5 example, a bit rate on the order of about 100 to 200 

Kbps, although the invention is not limited to any 
particular bit rate and is broad enough to encompass 
any desired bit rate that is low compared to that for 
normal (full motion) video. The invention is 
10 particularly well suited for digital television 

systems, and can be implemented, for example, in a 
standard (SDTV) or high definition television (HDTV) 
encoder . 

In accordance with the invention, still pictures 
15 are trickled out of the encoder over a relatively long 

period of time. As a result, transmission bandwidth 
is saved. In particular, the still pictures are 
provided in a manner analogous to a slide show, 
instead of as real-time video. Thus, rather than 
20 dedicating a full bandwidth real-time video service to 
still images, the still pictures can be sent to a 
population of decoders on a reduced bandwidth channel 
at, e.g., only 100 to 200 Kbits per second. Packets 
of the still picture data are provided under their own 
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program identifier (PID) within a conventional MPEG-2 
transport stream. 

In an MPEG-2 digital television application, an 
output bit rate on the order of about 100 to 200 Kbps 
5 translates to a video frame rate of about 1/10 to two 

frames per second, or a period of one-half to ten 
seconds per frame. As an example, an implementation 
may be provided where the longest period is ten 
seconds, assuming a rate of 100 Kbps, with a 1 Mbit 

10 output (i.e., compressed) frame size limitation 

(100Kbps * 10 sec. - 1 Mbit) . The MPEG MP@ML (main 
profile, main level) decoder has a 1.84 Mbit video 
buffer size. The decoder has about 1.5 frames in its 
video FIFO before it processes the frame, or 1.5 Mbits 

15 maximum buffer fullness at 100 Kbps, with a 10 second 
repetition rate. 

Present day digital television encoders, such as 
the DigiCipher® II Plus encoder available from the 
Broadband Communications Sector of Motorola, Inc., 

20 Horsham, Pennsylvania U.S.A., support still picture 

processing, whereby the user may wish to transmit 
still pictures at a low frame rate (e.g., 1/30 to two 
frames per second) . The user selects the following 
three parameters: 

25 1) Bit Rate (100Kbps - 200Kbps), 
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2) Still picture Capture Rate (rate at which a 
new input still picture is captured) , and 

3) Still picture Repetition Rate (rate at which 
still pictures are output) . 

5 During normal operation, frames are compressed at 

30, 29.97 or 24 frames/sec. using, e.g., the MPEG-2 
compression standard. MPEG-2 processes frames as I, 
P, or B (Intra, Predictive, or Bidirectional) 
depending on input parameters, N (the Group-of- 

10 Pictures (GOP) length) and M (the number of frames 

between anchor frames (i.e., I or B frames)). If the 
bit rate selected is 200 Kbps or less, still picture 
processing is automatically selected. 

With still picture processing in accordance with 

15 the present invention, the still pictures are encoded 

only as Intra (I) frames at the normal input frame 
rate of, e.g., 29.97 or 30 Hz (frames per second). 
However, not all of the encoded frames are transmitted 
to the decoder; only a subset of the encoded frames is 

20 transmitted at the still picture repetition rate, 

which is much lower than the input frame rate. 
Accordingly, still pictures are processed by the video 
compressor at the normal input frame rate, but 
communicated to the decoder at a much lower rate, such 

25 as 100 Kbps - 200 Kbps. 
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After the decoder has appoximately 1.5 compressed 
still picture frames in its video buffer, or the 
decoder video buffer is otherwise filled to a 
predetermined level, the decoder is required to 
5 process and display the next frame at a time specified 

by the encoder via a Presentation Time Stamp (PTS) . 
The PTS is chosen such that the entire frame will have 
entered the decoder buffer, and not too much data will 
arrive after this frame to cause the buffer to 

10 overflow. The criteria is not a predetermined buffer 

level, but a predetermined time. In the example 
implementation discussed above, a 1.5 frame times 
system delay was chosen to allow a degree of margin to 
prevent FIFO overflow or underflow. 

15 The use of a PTS, and associated Display Time 

Stamp (DTS) is well known in the art. Once received 
and decoded, the decoder will continuously display a 
received still picture at the frame rate, until a new 
compressed still picture is received. 

20 As illustrated in the still picture encoder of 

Figure 1, still pictures from a still frame input 
video stream 10 are received and stored in a current 
frame buffer 12 at the normal input frame rate. The 
still frame input video stream 10 is carried under a 

25 specific PID within a standard transport stream. The 
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normal input frame rate, at which the still pictures 
are provided in the video stream 10, is typically 
29*97 or 30 frames per second, although different 
standards may now or in the future provide other 
5 rates. A selector 16, which can be implemented using 

a standard multiplexer, is provided to enable 
selection between the current frame data stored in 
buffer 12 and previous frame data stored in buffer 14, 
for input to a video compressor 18. In the 

10 illustrated embodiment, the video compressor includes 

a conventional video compression engine 17, and an 
output gate 19. The output gate is controlled, as 
described in greater detail below, in order to provide 
compressed still pictures to a compressed video buffer 

15 20 at the desired frame repetition rate. It should be 

appreciated that the embodiment shown in Figure 1 is 
only one possible implementation of the invention, and 
that other implementations will be apparent to those 
skilled in the art. For example, the gate 19 could be 

20 located on the output side of the buffer 20, with 

compressed still pictures being overwritten in the 
buffer until a desired compressed picture has been 
buffered for output. 

Current still frames are captured from the 

25 current frame buffer at a "still picture capture rate" 
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which is less than the input frame rate. Between 
successive current frame captures, previous frames are 
provided to the video compressor 18 from the previous 
frame buffer 14. The captured current still picture 
5 frames and stored previous frames are successively 

provided to the video compressor at the normal video 
frame rate, so that the compression engine 17 can 
operate without modification. 

In the illustrated embodiment, the capture rate 

10 is converted to units of input frames, e.g., a frame 

capture period, wherein one frame is captured during 
each period. For example, if the converted capture 
rate (cap_rate) is ten frames, the current frame 
buffer 12 is selected once every ten input frame 

15 times. Otherwise, the previous frame buffer 14 is 

selected by selector 16. This is denoted by the 
mathematical expression frame_cnt % cap_rate, where 
"%" denotes the modulo operation, and frame_cnt is a 
count of all input frames. It should be appreciated 

20 that other implementations can be devised within the 

scope of the invention for controlling the selector 
16. 

All captured current frames and previous frames 
output by the selector 16 are compressed by the video 
25 compressor 18 as Intra frames. Therefore, MPEG-2 
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picture parameters N and M are set as: N=l, M=l for 
the video sequence. Moreover, in order to maintain 
the operation of the video compression engine 17 the 
same for still pictures as for normal video, all of 
5 the frames provided to the compressor are compressed 
at the normal input frame rate. However, in the 
illustrated embodiment, only frames at a still picture 
repetition rate (rep_rate) are stored in the 
compressed video buffer 20 for subsequent output to a 

10 transport processor 22. The storage of frames at the 

rep_rate is controlled by gate 19 in response to a 
control signal which actuates the gate to write data 
into buffer 2 0 when ( (frame__cnt + compressor_delay) % 
rep_rate) = 0. The term " compressor_delay M accounts 

15 for the inherent processing delay of the compression 

engine 17. 

Alternatively, all of the compressed frames from 
video compressor 18 can be written into compressed 
video buffer 20, with only a portion of those being 

20 delivered by the buffer to the transport processor 22. 

In either embodiment, the still picture repetition 
rate is the rate at which still pictures are 
communicated to the decoder. As indicated above, this 
rate (e.g., 100 to 200 Kbps) is much lower than the 

25 input frame rate at which the input video 10 is 
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received by the current frame buffer 12. Even though 
the present invention allows the rate at which still 
frames are transmitted to the decoder to be much lower 
than the rate at which such frames are received by the 
5 encoder, the video compression engine 17 continues to 

operate at its standard speed. 

In the illustrated embodiment, the still picture 
repetition rate is converted to units of input frames. 
For example, if the converted repetition rate is ten 

10 frames, e.g., a frame repetition period, one of every 

ten compressed pictures output from the compressor 18 
is stored in the compressed video buffer, i.e., one 
picture every ten input frame times. 

The frames stored in the compressed video buffer 

15 20 are selected such that if the capture rate is a 

multiple of the repetition rate, a current frame 
captured at the input (e.g., by buffer 12) and 
selected by selector 12 is the same frame stored in 
the FIFO 20. This is useful if the input frame 

2 0 changes at a higher rate than the capture rate. When 

the input frame image changes at a higher rate than 
the capture rate, not all of the images will be 
transmitted. Instead, only the images captured at the 
capture rate will be transmitted. 
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The capture rate and the repetition rate can also 
be the same (i.e., capture rate = repetition rate). 
In this event, the video is encoded at the input frame 
rate but, unlike the normal MPEG-2 encoding process, 
5 all frames are encoded as I-frames. Selected 

compressed frames are then sent to the decoder at the 
repetition rate. Thus, for example, if the repetition 
rate is one out of every ten frames, all ten frames 
will be compressed, but only one of each ten 

10 compressed frames will be sent to the decoder via the 

transport processor. The remaining frames are either 
not written into the compressed video buffer 20, 
overwritten in the video buffer, or dumped. In the 
embodiment illustrated in Figure 1, the first option 

15 is implemented via the control signal input to gate 

19, which only writes one of the ten frames into the 
buffer 20. It should be appreciated that the 
repetition rate described above is only an example, 
and that any ratio of transmitted frames to compressed 

2 0 frames can be used depending on the system 

requirements . 

As noted above, as the input video is received, 
it is written into the current frame buffer 12 and 
video frames are selected by selector 16 at the frame 

25 capture rate. In between capture rate cycles, the 
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video compressor receives frames to compress from the 
previous frame buffer 14 via the selector 16, which 
runs at the input frame rate so that the compression 
engine 17 is kept supplied with frames at its normal 
5 processing rate. The previous frame buffer simplifies 
the operation of the system and tracking of the 
frames, particularly if the capture and repetition 
rates are not the same. 

Advantageously, the rate control algorithm of the 

10 compression engine does not have to be changed from 

its normal operation in order to practice the present 
invention. This algorithm merely uses the repetition 
rate for the required frame rate parameters. At the 
same time, the desired output bit rate is used for the 

15 required bit rate parameters. The purpose of the rate 

control algorithm is to maintain the number of output 
bits within an available bandwidth. Since the present 
invention only transmits frames at the low repetition 
rate, the rate control uses the repetition rate in 

20 calculating the necessary bandwidth parameters. The 

input rate to the compression engine (i.e., the input 
video frame rate) does not matter for purposes of the 
rate control algorithm. 

Frames are inserted into the MPEG-2 transport by 

25 the transport processor 22. The PTS/DTS is set such 
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that, for example, 1.5 frames are stored in the 
decoder video buffer (e.g., a video FIFO) before the 
PTS/DTS time. This provides enough buffer time to 
ensure that a complete frame has entered the decoder 
5 video FIFO before the decoder is required to process 
that frame. 

Figure 2 illustrates an alternative embodiment of 
the invention which does not require the selector 16 
or the current and previous frame buffers 12 and 14, 

10 respectively. In this embodiment, raw video (the 

"input video") is provided to the compression engine 
17 at the video frame rate of, e.g., 30 or 29.97 
frames per second. The compression engine encodes all 
of the frames it receives. Those frames that are to 

15 be stored in the video memory 30, 32 for subsequent 

transmission to the decoders (via transport processor 
22) are encoded as I- frames. For simplicity, all of 
the frames received by the compression engine may be 
coded as I- frames. The video memory is a type which 

20 has a controllable read pointer, which can be reset to 

the beginning of a picture. In this manner, the same 
picture can be re-read from the memory several times. 

Upon initialization, the first picture to be 
captured from the input video is written into MEM-A 

25 30. The next captured picture will be written into 
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MEM-B 32. The memory used for picture storage toggles 
for each captured picture. The pictures must be 
double buffered in this manner, so that the transport 
processor 22 can read from one memory bank (30 or 32}, 
5 while the compression engine 17 is writing into the 

other memory bank. It should be appreciated that 
instead of the two separate memories 30, 32 
illustrated, one circular memory buffer with two read 
and two write pointers can be used. Such a circular 
10 buffer should be large enough to handle worst case 

conditions to avoid overwriting data before it is 
read. 

Also at initialization, the transport processor 
22 reads data from MEM-A 30 when the first picture is 

15 available. The transport processor reads data at the 

output bit rate. After the first picture has been 
read out, the transport processor will wait for the 
still picture repetition rate (rep__rate) period to 
expire. The rep_rate period starts at the time the 

20 first picture is read. When the rep_rate period 

expires, the transport processor reads that same 
picture again from MEM-A 30. This is repeated until a 
new captured picture is stored into MEM-B 32. After a 
new picture is stored in MEM-B, the transport 

25 processor commences reading from MEM-B when the 
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current rep_rate period expires. Reading of the new 
picture continues from MEM-B according to the same 
procedure noted above, until the next new captured 
picture is stored into MEM- A . In the event that a new 
5 rep__rate period occurs before the transport processor 

finishes reading the current picture, the transport 
processor immediately starts reading the same picture 
again, or starts reading the picture in the other 
memory (MEM-A or MEM-B) if a new capture frame has 

10 arrived. Thus, it can be seen that still frame 

pictures are continuously provided at the still 
picture repetition rate from MEM-A and MEM-B in a 
cyclical manner, where the cycles depend on the rate 
at which new pictures are captured from the output of 

15 the compression engine. 

It should now be appreciated that the present 
invention provides for the encoding and transmission 
of still picture images at a reduced frame rate and 
reduced transmission rate. Such a scheme is 

20 particularly useful for applications that do not 

require lots of data to be delivered continuously, 
such as weather information and text information 
identifying content that is delivered via a digital 
decoder. Such content can comprise, for example, 

25 multimedia presentations, music, or the like. 
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Various alternative implementations of the 
invention can be practiced. In one alternate 
embodiment, the transport processor 22 can request I- 
frames from the video compressor, instead of being 
5 automatically provided with such frames by the 

compressed video buffer 20. In such an embodiment, 
the compression engine will produce the I-frame at a 
specified time. The transport processor will wait 
until the I-frame is encoded, and request that it be 

10 stored in a video buffer. 

In another alternate embodiment, the compressor 
can produce all I-frames (as described in connection 
with the illustrated embodiment) and store them in the 
compressed video buffer. The transport processor then 

15 dumps the unused compressed still picture frames and 

transmits the desired frame at the still picture 
repetition rate. The frame transmitted can be 
selected in either of two ways. These are: 1) the 
compressor can A tag' that frame when it is captured at 

20 the capture rate; the transport processor will then 

transmit the tagged frames; or 2) the transport 
processor can select the frames at the capture rate 
(i.e., the transport processor reads the compressed 
video buffer data and counts frames) . 
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An advantage of the embodiment just described is 
that there is no need to provide a gate 19 at the 
input or output side of the video buffer 20. However, 
the transport processor must be redesigned for this 
5 embodiment, and a large compressed video buffer is 

required to hold the I- frames. 

Although the invention has been described in 
connection with various preferred embodiments, it 
should be appreciated that various modifications and 
10 adaptations may be made thereto without departing from 
the scope of the invention as set forth in the claims. 



